Duration: 30 mins

<p style="font-family: helvetica,arial,sans-serif; font-size:1.6em;color:black; background-color: #DDDDDD; text-align:justify""> Level: Intermediate        Pre-requisite Skills: Python

New Business Location (Cafe or Restaurant)

As a Cafe, Restaurant or Bar, I am looking for commercial space in the City of Melbourne where I can open a new venue or extend my existing venue.

I would like to know where similar businesses are located and the density of residents and office workers in comparison.

I want to know the number of seats I should provide based on seating capacity at other similar establishments in the same area and understand how this may have changed over time.

If on street or off-street parking is available nearby this may help me decide where to locate my venue.

  What this Use Case will teach you

At the end of this use case you will:

• understand what CLUE data is and how to access it
• explored a dataset derived from the CLUE survey
• learnt how to visualise CLUE data using different mapping visualisation techniques

  A brief introduction to CLUE data

The City of Melbourne conducts a bi-annual comprehensive survey of its residents and businesses called the "Census of Land Use and Employment (CLUE)". CLUE captures key information on land use, employment, and economic activity across the City of Melbourne.

CLUE datasets are a valuable tool for businesses looking to invest in the City of Melbourne and researchers wanting to understand those factors that influence and shape the dynamics of Australia's second largest metropolis and one of the world's most liveable cities.

CLUE data assists the City of Melbourne's business planning, policy development and strategic decision making. Investors, consultants, students, urban researchers, property analysts, businesses and developers can take advantage of CLUE to understand customers, the marketplace and the changing form and nature of the city.

Source: CLUE ( https://data.melbourne.vic.gov.au/stories/s/CLUE/rt3z-vy3t?src=hdr )

This use case makes extensive use of various CLUE datasets to illustrate the value to Data Scientists, Researchers and Software Developers.

CLUE Geospatial Data

CLUE Data is often coded to a specific location (Latitude and Longitude) and/or to a City precinct, referred to as the "CLUE small area". Datasets may also include the individual city block within a precinct referred to by its CLUE Block ID.

The geospatial coordinates describing these areas as polygons can be downloaded in GeoJSON format and used to show shaded areas on a map, known as a choropleth map. This can be a useful technique for illustrating broad trends or statistics for a city area rather than a specific location.

A map visualisation of CLUE Blocks and small areas can be found at the following links:

• CLUE small areas
• CLUE Blocks

  Which CLUE data should I use?

To begin our exploration and use of CLUE data we shall first import the necessary libraries to support our exploratory data analysis and visualisation.

Following are core packages required for this exercise.

The sodapy package is required specifically for accessing open data from SOCRATA compliant open data web sites.

The plotly.express package lets use build interact maps using map box services.

import os
import time

from datetime import datetime
import numpy as np
import pandas as pd
from sodapy import Socrata
import plotly.express as px

To connect to the Melbourne Open Data Portal we must establish a connection using the sodapy library by specifying a domain, being the website domain where the data is hosted, and an application access token which can be requested from the City of Melbourne Open Data portal by registering here (https://data.melbourne.vic.gov.au/signup).

For this exercise we will login anonymously.

apptoken = os.environ.get("SODAPY_APPTOKEN") # Anonymous app token
domain = "data.melbourne.vic.gov.au"
client = Socrata(domain, apptoken) # Open Dataset connection

WARNING:root:Requests made without an app_token will be subject to strict throttling limits.

Next, we will look at one of the CLUE datasets to better understand its structure and how we can use it.

Our data requirements from this use case include the following:

• Number of Residential Dwellings per CLUE Block
• Number of Employees per CLUE Block
• Number of Seats (Indoor and Outdoor) per Venue and CLUE Block
• Number of Off-Street parking spaces per CLUE Block

For this exercise, we shall start by examining the Residential Dwelling dataset. Each dataset in the Melbourne Open Data Portal has a unique identifier which can be used to retrieve the dataset using the sodapy library.

The Residential Dwelling dataset unique identifier is rm92-h5tq. We will pass this identifier into the sodapy command below to retrieve this data.

This dataset is placed in a Pandas dataframe and we will inspect the first three rows.

# Retrieve the "CLUE Residential Dwellings 2020" dataset
data_rm92_h5tq = pd.DataFrame.from_dict(client.get_all('rm92-h5tq'))

print(f'The shape of dataset is {data_rm92_h5tq.shape}.')
print('Below are the first few rows of this dataset:')

# Transpose the DataFrame for easier visual comparison. 
data_rm92_h5tq.head(3).T

The shape of dataset is (10402, 10).
Below are the first few rows of this dataset:

	0	1	2
census_year	2020	2020	2020
block_id	1	1	11
pbs_property_id	611394	611395	103957
bps_base_id	611394	611395	103957
street_name	545-557 Flinders Street MELBOURNE VIC 3000	561-581 Flinders Street MELBOURNE VIC 3000	517-537 Flinders Lane MELBOURNE VIC 3000
clue_small_area	Melbourne (CBD)	Melbourne (CBD)	Melbourne (CBD)
dwelling_type	Residential Apartments	Residential Apartments	Residential Apartments
dwelling_number	196	189	26
x_coordinate	144.9565145	144.9559094	144.9566569
y_coordinate	-37.82097941	-37.82108687	-37.81987147

We can see that there are 10,402 records and 10 fields describing each record.

We can see that each record show us the number of dwellings for each individual property and the type of dwelling e.g. House/Townhouse, Residential Apartments, etc.

The location of each property is given using:

• Latitude and Longitude
• CLUE Small Area and Block ID
• Property Id

The Census Year that the data was collected is also shown.

For our analysis of this dataset and others we will be restricting our analysis to the 2020 CLUE Census and summarising the data to CLUE Block level.

  Summarising Residential Dwelling data

We want to plot the density of both residential dwellings and employment at city block level rather than specific property or address and so should visualise this data as a choropleth map.

Let's start by summarising the data at CLUE small area and Block level.

Note: We include CLUE Small Area as one of our group by fields so we can display the CLUE Small area name in the popup window when you hover over the area on the map.

We want to summarise the data by summing the number of dwellings across all rows in the same CLUE Block.

The following cell creates a dataframe containing this summary of residential dwellings.

# Cast datatypes to correct type so we can summarise
data_rm92_h5tq[['census_year', 'dwelling_number']] = data_rm92_h5tq[['census_year', 'dwelling_number']].astype(int)
data_rm92_h5tq[['x_coordinate', 'y_coordinate']] = data_rm92_h5tq[['x_coordinate', 'y_coordinate']].astype(float)
data_rm92_h5tq = data_rm92_h5tq.convert_dtypes() # convert remaining to string
data_rm92_h5tq.dtypes

# create the aggregate dataset
groupbyfields = ['block_id','clue_small_area']
aggregatebyfields = {'dwelling_number': ["sum"]}

dwellingsByBlock = pd.DataFrame(data_rm92_h5tq.groupby(groupbyfields, as_index=False).agg(aggregatebyfields))

# Dataframse Group by creates two levels of headings
# so we flatten the headings to make it easier to extract data for plotting
dwellingsByBlock.columns = dwellingsByBlock.columns.map(''.join) # flatten column header
dwellingsByBlock.rename(columns={'clue_small_area': 'clue_area'}, inplace=True) #rename to match GeoJSON extract
dwellingsByBlock.rename(columns={'dwelling_numbersum': 'dwelling_count'}, inplace=True)
dwellingsByBlock.head(5)

	block_id	clue_area	dwelling_count
0	1	Melbourne (CBD)	385
1	101	West Melbourne (Residential)	863
2	103	Melbourne (CBD)	638
3	104	Melbourne (CBD)	1093
4	105	Melbourne (CBD)	1729

  Visualising Residential Dwelling on a Choropleth Map

We use the Plotly Python Open Source Graphing Library to generate maps from mapbox.

Creating a choropleth map requires us to know the geometry(shape) of each CLUE Block area as a collection of latitude and longitude points defining a polygon. This data can be downloaded from the Melbourne Open Data Portal in GeoJSON format.

We also need to supply the data to be used to highlight the CLUE Blocks and that data must include the same unique identifier for each Block contained in the GeoJSON data set.

Below we extract the Melbourne CLUE Block polygons into a JSON datatype.

The final line in the cell displays the unique key for each polygon which must also exist in the Residential Dwelling dataset.

from urllib.request import urlopen
import json

geoJSON_Id = 'aia8-ryiq' # Melbourne CLUE Block polygons in GeoJSON format

GeoJSONURL = 'https://'+domain+'/api/geospatial/'+geoJSON_Id+'?method=export&format=GeoJSON'
with urlopen(GeoJSONURL) as response:
    block = json.load(response)
    
block["features"][0]['properties'].keys()

dict_keys(['block_id', 'clue_area'])

Now using just one function call called 'choropleth_mapbox' we can diaplay an interactive map visualisation using the block GeoJSON data to define the regions and the dwellingsByBlock dataframe to define the summarised data by block.

# Set the colour range to the maximum number of dwellings.
range_max = dwellingsByBlock['dwelling_count'].max()

# Display the choropleth map
fig = px.choropleth_mapbox(dwellingsByBlock, # pass in the summarised dwellings per block
                           geojson=block, # pass in the GeoJSON data defining the CLUE Block polygons
                           locations='block_id', # define the unique identifier for the Blocks from the dataframe
                           color='dwelling_count', # change the colour of the block region according to the dwelling count
                           color_continuous_scale=["#FFFF88", "yellow", "orange", "orange",
                                                   "orange", "darkorange", "red", "darkred"], # define custom colour scale
                           range_color=(0, range_max), # set the numeric range for the colour scale
                           featureidkey="properties.block_id", # define the Unique polygon identifier from the GeoJSON data
                           mapbox_style="stamen-toner", # set the visual style of the map
                           zoom=12.5, # set the zoom level
                           center = {"lat": -37.813, "lon": 144.945}, # set the map centre coordinates
                           opacity=0.5, # opacity of the choropleth polygons
                           hover_name='clue_area', # the title of the hover pop up box
                           hover_data={'block_id':True,'dwelling_count':True}, # defines which dataframe fields to display
                                                                               # in the hover popup box
                           labels={'dwelling_count':'Number of Dwellings','block_id':'CLUE Block Id'}, # defines labels for
                                                                               # the hover popup box
                           title='Residential Dwellings by CLUE Block Id for 2020', # Title for plot
                           width=950, height=800 # dimensions of plot in pixels
                          )
fig.show()

Congratulations!

You've successfully used Melbourne CLUE Open Data and Plotly to visualise residential density in the City of Melbourne!

Now zoom in and out on the map above to explore the city and areas of high and low residential density.

This is your first step to selecting a suitable location for your new business!

You can explore the Residential Density data here.

  Further sections (WIP)

  Interactive Visualisation: Melbourne Business Locator Tool

This interactive visualisation uses CLUE data to let users explore opportunities for locating your hospitality business in the City of Melbourne.